Integrated estimation of UAV image orientation with a generalised building model

The estimation of position and attitude of a camera, addressed as image orientation in photogrammetry, is an important task to obtain information on where a platform is located in the world or relative to objects. Unmanned aerial vehicles (UAV) as an increasingly popular platform led to new applications, some of which involve low flight altitudes and specific requirements such as low weight and low cost of sensors. Image orientation needs additional information to retrieve not only relative measurements but position and attitude in a world coordinate system. Given the requirements on sensors and especially for flights in between obstacles in urban environments classically used information of Global Navigation Satellite Systems (GNSS) and Inertial Measurement Units (IMU) or specially marked ground control points (GCP) are often inaccurate or unavailable. The idea addressed within this thesis is to improve the UAV image orientation based on an existing generalised building model. Such models are increasingly available and provide ground control that is helpful to compensate inaccurate or unavailable camera positions measured by GNSS and drift effects of image orientation. Typically, for UAV applications in street corridors, the geometric accuracy and the level of detail of such models is low compared to the high accuracy and high geometric resolution of the image measurements. Therefore, although the building model differs from the observed scene due to its generalisation, relations of the photogrammetric measurements to the building model are formulated and used in the determination of image orientation. Three approaches to assign tie points to model planes in object space are presented, and a sliding window as well as a global hybrid bundle adjustment are set up for image orientation aided by a generalised building model. The assignments lead to fictitious observations of the distance of tie points to model planes and are iteratively refined by bundle adjustment. Experiments with an image sequence captured flying between buildings show an improvement of image orientation from the metre range with purely GNSS measurements to the decimetre range when using the generalised building model with the simplest assignment method based on point-to-plane distances. No improvement by searching planes in the tie point cloud to indirectly find the relations of tie points to model planes is observed. The results are compared to a building model of higher detail and systematic effects are investigated. In summary, the developed method is found to significantly improve UAV image orientation using a generalised building model successfully.Die Schätzung von Position und Lage einer Kamera, die in der Photogrammetrie als Bildorientierung bezeichnet wird, ist eine grundlegende Aufgabe, um Informationen darüber zu erhalten, wo sich eine Plattform in der Welt oder relativ zu Objekten befindet. Zunehmend führen unbemannte Luftfahrtsysteme (UAV) als Plattform zu neuen Anwendungen, die zum Teil geringe Flughöhen und spezifische Anforderungen wie Gewicht und Kosten der Sensoren mit sich bringen. Für die Bildorientierung werden zusätzliche Informationen benötigt, um nicht nur relative Messungen, sondern auch Position und Lage in einem Weltkoordinatensystem bestimmen zu können. Angesichts dieser Anforderungen und insbesondere für Flüge zwischen Hindernissen in städtischen Gebieten sind die klassisch verwendeten Informationen von Navigationssatelliten- (GNSS) und Intertialmesssystemen (IMU) oder auch speziell markierten Passpunkten (GCP) oft nicht verfügbar oder zu ungenau. Die hier behandelte Idee ist daher, die Bildorientierung von UAVs auf der Grundlage eines bestehenden generalisierten Gebäudemodells zu verbessern. Solche Modelle sind in zunehmendem Maße verfügbar und bieten eine Möglichkeit, ungenaue oder nicht verfügbare GNSS-Kamerapositionen und Drifteffekte der Bildorientierung zu kompensieren. Bei UAV-Befliegungen in Straßenschluchten sind die geometrische Genauigkeit und der Detaillierungsgrad solcher Modelle im Vergleich zur hohen Genauigkeit und hohen geometrischen Auflösung der Bildmessungen typischerweise gering. Obwohl das Modell also aufgrund seiner Generalisierung von der beobachteten Szene abweicht, können Beziehungen der photogrammetrischen Messungen zum Gebäudemodell formuliert und in der Bildorientierung verwendet werden. Es werden drei Ansätze zur Zuordnung von Verknüpfungspunkten zu Modellebenen im Objektraum sowie eine hybride Bündelausgleichung zur Bildorientierung mit Hilfe eines generalisierten Gebäudemodells, die global oder fensterbasiert abläuft, vorgestellt. Die Zuordnungen führen zu fiktiven Beobachtungen für den Abstand von Verknüpfungspunkten zu Modellebenen und werden während der iterativen Bündelausgleichung verfeinert. Experimente mit einer zwischen Gebäuden aufgenommenen Bildsequenz zeigen eine Verbesserung der Bildorientierung vom Meterbereich rein mit GNSS-Messungen in den Dezimeterbereich bei Verwendung des generalisierten Gebäudemodells mit der einfachsten Zuordnungsmethode auf Basis von Punkt-zu-Ebene-Distanzen. Eine Verbesserung der Punkt-zu-Ebene-Zuordnungen durch die Suche von Ebenen in der Punktwolke wird nicht beobachtet. Zusammenfassend lässt sich sagen, dass die entwickelte Methode die UAV-Bildorientierung mit Hilfe eines generalisierten Gebäudemodells signifikant verbessert

Higher education institutions and knowledge triangle: improving the interaction between education, research and innovation

A critical review of the feasibility of the concept of the knowledge triangle (KT) as a basis for policy is presented. The research shows a gap between policy discourses and academic research. KT appears as a policy driven concept with superficial plausibility, however, has not been much analysed and evaluated. As a concept for policy making the KT seems complex and poorly understood. Few concrete approaches of the KT were observable (the European Institute of Innovation & Technology EIT and the more conceptual European Association of Institutions in Higher Education EURASHE concept). The analysis provides an analytical framework and proceeds by looking at the three two- way-relationships included, and then tries to draw extrapolations towards the three-way relationship indicated by the KT. A focus is the ‘Third Mission’ of universities, that has various, and partly contradictory meanings. A basic challenge is that the concept requires a turn from the ongoing differentiation process in higher education towards (re)-integration.Eine kritische Analyse der Literatur zur Tragfähigkeit des Konzepts des Knowledge Triangle (KT) als Basis für Politik wird präsentiert. Es zeigt sich eine Spaltung zwischen den politischen Diskursen und der Forschung. KT erscheint als oberflächliches politisches Konzept, zu dem es (noch) (fast) keine Forschungsergebnisse gibt, es wurden auch nur wenige Ansätze einer Realisierung gefunden. Auf Basis eines analytischen Rahmens werden die drei involvierten zweifach-Interaktionen im Dreieck untersucht, und es werden Extrapolationen auf die Dreifach-Interaktion vorgenommen. Einen Schwerpunkt bildet die ‚Third Mission‘ der Universitäten. Eine wesentliche Herausforderung für eine KT Politik besteht darin, dass diese die vorherrschende Tendenz der Differenzierung im Hochschulwesen zugunsten einer (Re)- Integration der Funktionen ‚umdrehen‘ muss

Georeferencing of an Unmanned Aerial System by Means of an Iterated Extended Kalman Filter Using a 3D City Model

In engineering geodesy, the technical progress leads to various kinds of multi-sensor systems (MSS) capturing the environment. Multi-sensor systems, especially those mounted on unmanned aerial vehicles, subsequently called unmanned aerial system (UAS), have emerged in the past decade. Georeferencing for MSS and UAS is an indispensable task to obtain further products of the data captured. Georeferencing comprises at least the determination of three translations and three rotations. The availability and accuracy of Global Navigation Satellite System (GNSS) receivers, inertial measurement units, or other sensors for georeferencing is not or not constantly given in urban scenarios. Therefore, we utilize UAS-based laser scanner measurements on building facades. The building latter are modeled as planes in a three-dimensional city model. We determine the trajectory of the UAS by combining the laser scanner measurements with the plane parameters. The resulting implicit measurement equations and nonlinear equality constraints are covered within an iterated extended Kalman filter (IEKF). We developed a software simulation for testing the IEKF using different scenarios to evaluate the functionality, performance, strengths, and remaining challenges of the IEKF implemented

Assigning tie points to a generalised building model for UAS image orientation

This paper addresses the integration of a building model into the pose estimation of image sequences. Images are captured by an Unmanned Aerial System (UAS) equipped with a camera flying in between buildings. Two approaches to assign tie points to a generalised building model in object space are presented. A direct approach is based on the distances between the object coordinates of tie points and planes of the building model. An indirect approach first finds planes within the tie point cloud that are subsequently matched to model planes; finally based on these matches, tie points are assigned to model planes. For both cases, the assignments are used in a hybrid bundle adjustment to refine the poses (image orientations). Experimental results for an image sequence demonstrate improvements in comparison to an adjustment without the building model. Differences and limitations of the two approaches for pointplane assignment are discussed - in the experiments they perform similar with respect to estimated standard deviations of tie points

The temporal build-up of hummingbird/plant mutualisms in North America and temperate South America

Background: Global Plants, a collaborative between JSTOR and some 300 herbaria, now contains about 2.48 million high-resolution images of plant specimens, a number that continues to grow, and collections that are digitizing their specimens at high resolution are allocating considerable recourses to the maintenance of computer hardware (e.g., servers) and to acquiring digital storage space. We here apply machine learning, specifically the training of a Support-Vector-Machine, to classify specimen images into categories, ideally at the species level, using the 26 most common tree species in Germany as a test case. Results: We designed an analysis pipeline and classification system consisting of segmentation, normalization, feature extraction, and classification steps and evaluated the system in two test sets, one with 26 species, the other with 17, in each case using 10 images per species of plants collected between 1820 and 1995, which simulates the empirical situation that most named species are represented in herbaria and databases, such as JSTOR, by few specimens. We achieved 73.21% accuracy of species assignments in the larger test set, and 84.88% in the smaller test set. Conclusions: The results of this first application of a computer vision algorithm trained on images of herbarium specimens shows that despite the problem of overlapping leaves, leaf-architectural features can be used to categorize specimens to species with good accuracy. Computer vision is poised to play a significant role in future rapid identification at least for frequently collected genera or species in the European flora

Safety and clinical outcomes of rituximab therapy in patients with different autoimmune diseases: experience from a national registry (GRAID)

Introduction: Evidence from a number of open-label, uncontrolled studies has suggested that rituximab may benefit patients with autoimmune diseases who are refractory to standard-of-care. The objective of this study was to evaluate the safety and clinical outcomes of rituximab in several standard-of-care-refractory autoimmune diseases (within rheumatology, nephrology, dermatology and neurology) other than rheumatoid arthritis or non-Hodgkin's lymphoma in a real-life clinical setting. Methods: Patients who received rituximab having shown an inadequate response to standard-of-care had their safety and clinical outcomes data retrospectively analysed as part of the German Registry of Autoimmune Diseases. The main outcome measures were safety and clinical response, as judged at the discretion of the investigators. Results: A total of 370 patients (299 patient-years) with various autoimmune diseases (23.0% with systemic lupus erythematosus, 15.7% antineutrophil cytoplasmic antibody-associated granulomatous vasculitides, 15.1% multiple sclerosis and 10.0% pemphigus) from 42 centres received a mean dose of 2,440 mg of rituximab over a median (range) of 194 (180 to 1,407) days. The overall rate of serious infections was 5.3 per 100 patient-years during rituximab therapy. Opportunistic infections were infrequent across the whole study population, and mostly occurred in patients with systemic lupus erythematosus. There were 11 deaths (3.0% of patients) after rituximab treatment (mean 11.6 months after first infusion, range 0.8 to 31.3 months), with most of the deaths caused by infections. Overall (n = 293), 13.3% of patients showed no response, 45.1% showed a partial response and 41.6% showed a complete response. Responses were also reflected by reduced use of glucocorticoids and various immunosuppressives during rituximab therapy and follow-up compared with before rituximab. Rituximab generally had a positive effect on patient well-being (physician's visual analogue scale; mean improvement from baseline of 12.1 mm). Conclusions: Data from this registry indicate that rituximab is a commonly employed, well-tolerated therapy with potential beneficial effects in standard of care-refractory autoimmune diseases, and support the results from other open-label, uncontrolled studies

Intestinal B-cells license metabolic T-cell activation in NASH microbiota/antigen-independently and contribute to fibrosis by IgA-FcR signalling

BACKGROUND & AIMS The progression of nonalcoholic steatohepatitis (NASH) to fibrosis and hepatocellular carcinoma (HCC) is aggravated by auto-aggressive T cells. The gut-liver axis contributes to NASH, but the mechanisms involved and the consequences for NASH-induced fibrosis and liver cancer remain unknown. We investigated the role of gastrointestinal B cells in the development of NASH, fibrosis and NASH-induced HCC. METHODS C57BL/6J wild-type (WT), B cell-deficient and different immunoglobulin-deficient or transgenic mice were fed distinct NASH diets (for example, choline-deficient high-fat diet, CD-HFD) or chow diet for 6 or 12 months, whereafter NASH, fibrosis, and NASH-induced HCC were assessed and analysed. Specific pathogen-free/germ-free WT and μMT mice (containing B cells only in the gastrointestinal tract) were fed a CD-HFD, and treated with an anti-CD20 antibody, whereafter NASH and fibrosis were assessed. Tissue biopsy samples from patients with NAFL, NASH and cirrhosis were analysed to correlate the secretion of immunoglobulins to clinicopathological features. Flow cytometry, immunohistochemistry and scRNA-Seq analysis were performed in liver and gastrointestinal tissue for immune cells in mice and humans. RESULTS Activated intestinal B cells were increased in mouse and human NASH samples and licensed metabolic T-cell activation to induce NASH independently of antigen-specificity and gut microbiota. Genetic or therapeutic depletion of systemic or gastrointestinal B cells prevented or reverted NASH and liver fibrosis. IgA secretion was necessary for fibrosis induction by activating CD11b+CCR2+F4/80+CD11c-FCGR1+ hepatic myeloid cells through an IgA-FcR signalling axis. Similarly, patients with NASH had increased numbers of activated intestinal B-cells and showed a positive correlation between IgA levels and activated FcRγ+ hepatic myeloid cells as well extent of liver fibrosis. CONCLUSIONS Intestinal B cells and the IgA-FcR signalling axis represent potential therapeutic targets for treating NASH. IMPACT AND IMPLICATIONS Nonalcoholic steatohepatitis (NASH) is a chronic inflammatory condition on the rise and can lead to hepatocellular carcinoma (HCC), the 3rd most common cause of cancer-related death worldwide. Currently, there is no effective treatment for this progressive disease that correlates with a marked risk of HCC mortality and carries a substantial healthcare burden. To date, among all the solid tumours, especially in HCC, the incidence and mortality rates are almost the same, making it crucial to find curative treatments for chronic diseases, such as NASH, which highly predispose to tumorigenesis. We have previously shown that NASH is an auto-aggressive condition aggravated, amongst others, by T cells. Therefore, we hypothesized that B cells might have a role in disease induction and progression. Our present work highlights that B cells have a dual role in NASH pathogenesis, being implicated in the activation of auto-aggressive T cells and the development of fibrosis via activation of monocyte-derived macrophages by secreted immunoglobulins (e.g., IgA). Furthermore, we could show that the absence of B cells prevented HCC development. B-cell intrinsic signalling pathways, secreted immunoglobulins, and interactions of B cells with other immune cells are potential targets in combinatorial NASH therapies against inflammation and fibrosis

An eV-scale sterile neutrino search using eight years of atmospheric muon neutrino data from the IceCube Neutrino Observatory

The results of a 3+1 sterile neutrino search using eight years of data from the IceCube Neutrino Observatory are presented. A total of 305,735 muon neutrino events are analyzed in reconstructed energy-zenith space to test for signatures of a matter-enhanced oscillation that would occur given a sterile neutrino state with a mass-squared differences between 0.01\,eV$^2$ and 100\,eV$^2$. The best-fit point is found to be at $\sin^2(2\theta_{24})=0.10$ and $\Delta m_{41}^2 = 4.5{\rm eV}^2$, which is consistent with the no sterile neutrino hypothesis with a p-value of 8.0\%.Comment: 11 pages, 5 figures. This letter is supported by the long-form paper "Searching for eV-scale sterile neutrinos with eight years of atmospheric neutrinos at the IceCube neutrino telescope," also appearing on arXiv. Digital data release available at: https://github.com/icecube/HE-Sterile-8year-data-releas